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Abstract 

Pilots’  mental  resources  can  be  overburdened  by  the  information  presented  in  cockpit  displays.  Previous 
research  found  that  presenting  information  in  a  3D  perspective,  or  virtual,  head-up  display  (HUD)  can  lessen 
information  overload.  The  current  research  examined  whether  extending  virtual  HUDs  into  the  visual 
periphery  will  further  reduce  the  mental  workload  associated  with  spatial  orientation  processes  such  as 
perception  and  control  of  the  heading  direction  and  speed  of  self-motion  (egospeed).  It  found  that:  a) 
heading  control  based  on  peripheral  vision  is  equivalent  to  that  based  on  central  vision  if  informative  areas 
of  optical  flow  are  visible;  b)  control  of  heading  requires  only  a  limited  amount  of  attentional  resources;  and 
c)  the  complex  motion  created  by  transparently  superimposing  optical  flow  in  a  display  can  lead  to 
systematic  heading  errors.  Additional  research  found  that  perception  and  control  of  egospeed  is  based 
primarily  on  the  mean  image  velocity  of  the  display,  but  that  texture  density  and  motion  parallax  produce 
small  but  consistent  biases.  Further  research  found  better  flight-path  and  speed  control  performance  with  a 
peripherally-located  virtual  speed-error  HUD  as  compared  to  the  MIL-STD-1787B  HUD  airspeed  indicator. 
Taken  as  a  whole,  these  results  suggest  that  peripheral  virtual  HUDs  have  significant  potential  for  conveying 
spatial  orientation  with  fewer  resource  demands  than  conventional  HUDs. 


Background  and  Research  Objectives 

To  successfully  complete  their  missions,  pilots  must 
quickly  perceive  and  analyze  a  vast  amount  of  information. 
Increasingly  complex  aircraft,  weapon  systems,  and  mission 
requirements  combine  to  create  a  level  of  workload  that  may 
overwhelm  the  perceptual  and  cognitive  resources  of  pilots, 
thereby  reducing  pilot  performance  and  increasing  the  risk  of 
accidents.  Modem  cockpit  displays  that  present  navigational 
and  tactical  information  in  an  integrated,  3D  perspective, 
head-up  display  (HUD)  format — what  we  will  subsequently 
refer  to  as  a  virtual  HUD — have  been  shown  to  moderate 
workload  while  simultaneously  increasing  performance  and 
reducing  the  risk  of  accidents  (cf.  Hettinger,  Brickman,  Roe, 
Nelson,  &  Haas,  1996). 

However,  because  these  advanced  displays  present 
information  to  the  central  visual  field  only,  some  problems 
remain.  For  example,  the  superposition  of  HUD  or  helmet- 
mounted  display  (HMD)  symbology  over  the  pilot’s  view  of 
the  outside  world  can  lead  to  cognitive  capture  and  the 
potential  that  important  information,  either  within  the  HUD  or 
outside  the  window  may  be  obscured  (Boston  &  Braun,  1996; 
Becklen  &  Cervone,  1984).  Furthermore,  competition  within 
central  vision  among  tasks  requiring  focused  attention,  such  as 
target  acquisition  and  flight  systems  monitoring,  may  interfere 
with  pilots’  spatial  orientation — the  zero-  and  higher-order 
time  derivatives  of  a  pilot’s  position  and  orientation  in 
space — leading  to  reduced  situation  awareness. 

One  potential  solution  to  these  problems  is  to  present 
selected  visual  information  to  the  pilot  with  displays  located  in 
the  visual  periphery  and  thus  take  advantage  of  the  relatively 
underused  peripheral  visual  field  (Roscoe,  1980).  Some  tasks 


may  be  especially  well  suited  for  peripheral  displays.  For 
example,  psychophysical  research  has  shown  that  the 
peripheral  visual  field  can  provide  superior  performance  on 
tasks  related  to  spatial  orientation  (Brandt,  Dichgans,  & 
Koenig,  1973;  Dichgans,  Held,  Young,  &  Brandt,  1972; 
Dichgans  8l  Brandt,  1974;  Held,  Dichgans,  &  Bauer,  1975; 
Lestienne,  Soechting,  &  Berthoz,  1977;  Stoffregen,  1985). 
This  report  summarizes  further  research  that  examined  the 
potential  of  peripheral  visual  displays  for  spatial  orientation 
during  vehicular  control,  including  experiments  conducted 
within  an  ecologically  valid  dual-task  context  where 
attentional  and  central  visual  field  resources  are  demanded  by 
other  tasks. 

There  were  three  general  objectives  of  this  research.  The 
first  objective  was  to  develop  the  technical  capacity  for 
examining  human  visual  performance  in  the  context  of  active 
control.  Due  to  the  novelty  of  this  research  area,  much  of  the 
software  necessary  to  display  interactive  simulations  with 
precise  measurement  and  analysis  needed  to  be  developed.  A 
second  objective  was  to  determine  the  relative  capability, 
information  basis,  and  resource  demands  of  perception  and 
control  of  spatial  orientation  as  a  function  of  visual  field.  Two 
aspects  of  spatial  orientation  were  examined:  a)  perception  of 
egospeed,  the  speed  of  self-motion;  and  b)  perception  of 
heading,  the  direction  of  self-motion.  A  third  objective  was  to 
determine  the  optimal  format  for  peripherally-located  virtual 
HUDs  and  to  test  these  displays  against  currently-used  display 
formats.  Based  on  design  principles  derived  fi*om 
psychophysical  research  on  perceived  egospeed,  a 
peripherally-located  speed-error  display  was  developed  and 
compared  to  the  military  standard  (MIL-STD-1787B)  speed 
display  in  a  dual-task  environment  in  which  observers 
controlled  both  their  flight-path  and  speed. 
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Technical  Summary  of  Significant  Work 
Accomplished 

Research  Software  Development 

VIEWER  (Virtual  Environment  Workbench  for  Education 
and  Research).  An  initial  objective  was  to  develop  the 
software  necessary  to  produce  the  visual  simulations  to  be 
used  in  experiments  involving  closed-loop  control  of 
locomotion  through  virtual  environments.  To  achieve  this 
objective,  ViEWER,  an  extension  of  the  OpenGL  graphics- 
library,  was  developed.  ViEWER  supports  the  design  and 
presentation  of  simulations  for  the  scientific  study  of  human 
interactions  with  virtual  environments.  In  contrast  to  existing 
commercial  software  packages  for  developing  virtual 
environments,  ViEWER  was  specifically  designed  for 
research,  combining  flexible  simulation  of  many  types  of 
virtual  environments,  precise  timing  of  animation,  and  the 
ability  to  precisely  measure  human  performance  within  virtual 
environments.  ViEWER  vl  .03  has  a  number  of  important 
features.  First,  since  it  is  based  on  Silicon  Graphics’  OpenGL 
graphics  library,  it  can  potentially  be  used  on  graphics 
workstations  running  either  Microsoft  Windows  (95,  98,  me, 
NT4,  or  2000)  or  UNIX  (IRIX,  AIX,  HP  UX,  Solarix,  or 
Linux)  operating  systems.  Second,  the  software  allows 
precise  control  of  timing  and  animation,  making  it  ideal  for 
research  applications  that  depend  on  precise  temporal  control 
and  measurement.  Third,  the  software  supports  the 
development  of  a  variety  of  virtual  environments,  both 
realistic  (e.g.,  flight  simulation)  and  abstract  (e.g.,  dot  displays 
used  in  typical  experiments  in  vision  science).  Fourth,  the 
software  allows  users  to  specify  what  types  of  measures  will 
be  recorded.  These  measures  currently  include  tracking  of 
participant  movements  through  the  virtual  environment,  user 
inputs  from  controls  such  as  joysticks  and  footpedals,  and 
physiological  measurements  such  as  eye  and  head  movements. 
Presently,  ViEWER  vl.03  is  usable  by  those  with  modest  C++ 
programming  knowledge,  allowing  researchers  to  develop 
virtual  environments  in  a  fraction  of  the  time  needed  to 
program  these  environments  from  scratch. 

FUSE  (Front-end  Utility  for  Setting-up  Experiments). 
FUSE  is  an  easy-to-use  utility  that  allows  researchers  to 
design  experiments  that  make  use  of  the  interactive  virtual 
environments  developed  with  ViEWER.  With  FUSE  an 
experimenter  can  set  the  order  and  features  of  experimental 
conditions  (blocks  and  trials)  to  be  used  for  data  collection, 
and  can  specify  parameters  to  control  Fif^lTE'i^-generated 
simulations,  such  as  the  data  to  be  collected  or  external 
disturbances. 

Automated  NASA-TLX  Subjective  Workload  Scale.  This 
program  implements  the  widely  used  NASA-TLX  Subjective 
Workload  Scale  (Hart  &  Staveland,  1999)  in  an  easy-to-use 
software  tool  that  automates  data  collection  and  analysis. 

Time-Series  Analysis  Tool  (CLAnalyzer).  This  program 
allows  researchers  to  quickly  derive  useful  measures  from  the 
time-series  data  produced  by  ViEWER  simulations  for  further 
inferential  analysis  with  standard  statistical  programs. 


Research  on  Heading  Perception 

An  important  component  of  dynamic  spatial  orientation  is 
the  perception  of  the  direction  of  observer  motion  (the  heading 
direction).  An  accurate  perception  of  the  heading  direction  is 
particularly  important  for  visually  demanding  flight  missions 
such  as  terrain-following  or  nap-of-the-earth  (NOE)  flight,  in 
which  pilots  are  required  to  fly  at  low  altitudes  at  relatively 
high  speeds  (R.  Warren,  1988;  Kleiss  &  Hubbard,  1993),  and 
where  serious  consequences  may  result  from  misperceptions 
of  heading.  Currently,  pilots  rely  primarily  on  alpha-numeric 
displays  and  visual  information  available  “out-the-window” 
for  determining  heading  in  these  circumstances  (R.  Warren, 
1988).  However,  accurate  control  of  heading  could  be 
enhanced  through  the  use  of  a  peripherally-located  virtual 
display  comprised  of  flowing  elements  that  cue  the  direction 
of  heading  in  a  natural  manner.  Understanding  the  visual 
mechanisms  underlying  perception  of  heading  is  thus  an 
important  basic  perceptual  problem  that  has  potential 
application  to  virtual  display  design. 

Three  aspects  of  heading  perception  are  particularly 
relevant  to  the  design  and  use  of  peripherally-located  virtual 
displays  as  aids  for  spatial  orientation  and  visual  navigation: 

1)  the  capability  of  the  peripheral  visual  field  for  supporting 
control  of  heading,  2)  the  attentional  demands  of  heading 
control,  and  3)  the  effect  on  heading  perception  of 
superimposing  moving  symbols  in  a  virtual  HUD  or  HMD 
over  the  directly- vie  wed  scene.  Research  examining  these 
three  issues  is  summarized  below.  For  more  detailed 
information,  please  refer  to  the  theses  and  articles  listed  in  the 
section  titled,  “Publications  Stemming  from  the  Research 
Effort.” 

Peripheral  Vision  and  Heading  Performance.  Previous 
studies  examining  the  contributions  of  central  and  peripheral 
vision  (also  referred  to  as  retinal  eccentricity)  to  heading 
perception  did  not  assess  the  task  of  heading  perception  in  its 
most  common  everyday  form:  Where  an  observer  is  free  to 
look  over  a  large  field  of  view  while  controlling  their 
locomotion  over  periods  of  minutes  to  hours  (e.g.,  Atchley  & 
Andersen,  1997;  Crowell  &  Banks,  1993;  Warren  &  Kurtz, 
1992).  Rather,  these  studies  required  observers  to  fixate  their 
gaze  on  a  stationary  object  while  viewing  brief  and  small 
displays  simulating  self-motion  (e.g.,  lasting  less  than  1  s  in 
duration  and  presented  on  a  21”  computer  monitor  subtending 
perhaps  10-40  degrees  of  visual  angle)  and  then  respond  with 
a  verbal  judgment  of  their  perceived  direction  of  heading.  Our 
understanding  of  the  contribution  of  the  central  and  peripheral 
visual  fields  to  heading  perception  is  thus  limited  to  the 
special  case  of  an  observer  with  no  control — in  effect  a 
passenger — ^whose  eyes  are  fixed  in  a  particular  direction, 
viewing  the  world  through  a  small  window  for  a  brief  time 
period  and  then  making  a  verbal  judgment.  Clearly,  the 
general  conclusion  found  by  this  previous  research,  that  the 
central  visual  field  is  necessary  and  sufficient  for  optimal 
heading  performance,  does  not  necessarily  apply  to  the  more 
common  and  relevant  case  of  an  observer  engaged  in  closed- 
loop  control  (i.e,,  a  pilot),  who  is  freely  looking  throughout  a 
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Display  Geometry 


Figure  1.  Pictorial  representation  (top-down  view)  of  the  display 

geometry  used  to  present  the  90  x  34  degree  (H  x  V) 
field  of  view. 


large  field  of  view  as  they  control  their  direction  of  motion 
over  a  more  extended  time  period. 

The  experiments  reported  here  extend  our  understanding 
of  the  relationship  between  retinal  eccentricity  and  heading 
perception  to  the  more  ecologically  relevant  task  of  active 
control  of  heading.  The  method  used  in  these  experiments 
was  to  present  observers  30  s  simulations  of  flight  on  a 
straight  path  through  a  field  of  moving  points  (a  star  field). 
Simulated  wind  gusts  (defined  by  a  sum  of  sine  waves  at  five 
prime  frequencies)  rotated  the  observer’s  viewpoint  about 
either:  a)  the  vertical  (yaw)  axis  passing  directly  through  their 
eye,  as  if  the  observer  was  seated  in  a  turning  swivel  chair,  or 
b)  the  horizontal  (pitch)  axis  passing  from  left  to  right  through 
their  eye,  as  if  the  observer  was  seated  in  a  rocking  chair.  The 
simulations  were  presented  on  a  large,  90  x  34  degree  (H  x  V) 
display  formed  by  placing  two  rear-projection  screens  side-by- 
side  at  an  angle  of  135  degrees  (see  Figure  1).  The  observer’s 
task  was  to  control  their  direction  of  yaw  or  pitch  so  that  they 
countered  the  simulated  wind  disturbances  and  maintained 
their  perceived  direction  of  heading  straight-ahead  toward  the 
center  of  the  display  screens. 

Two  variables  were  of  primary  interest:  a)  the  field  of 
view  stimulated,  and  b)  the  axis  of  heading  perception  and 
control.  Field  of  view  was  varied  on  three  levels:  a)  central — 
within  a  1 5  degree  radius  circle  centered  on  the  direction  of 
gaze;  b)  peripheral — outside  a  15  degree  radius  circle  centered 
on  the  direction  of  gaze;  and  c)  full-field — ^the  entire  90  x  34 
degree  display  (see  Figure  2).  Unlike  previous  studies,  this 
manipulation  of  visual  field  did  not  require  observers  to  fixate 
their  direction  of  gaze  on  a  particular  point  or  object  in  the 
display.  Rather,  observers  were  free  to  look  anywhere  in  the 
display  and  wore  an  eye-head  tracking  apparatus  that  allowed 
their  direction  of  gaze  to  be  monitored  in  real-time.  Using  this 
measurement  of  gaze  direction,  the  graphics  computer 
generating  the  displays  yoked  a  mask  or  aperture  to  the  gaze 
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Figure  2.  Pictorial  representation  of  the  optical  flow  fields  used  to 

examine  the  relationship  between  heading  performance 
and  retinal  field  of  view.  Each  arrow  represents  the 
velocity  of  a  moving  dot.  Note:  the  mask  or  aperture 
was  centered  on  the  gaze  direction  and  moved  with  it 
across  the  displays. 

direction,  which  constrained  the  moving  dots  to  appear  only  in 
the  central  or  peripheral  visual  fields.  In  effect,  this  simulated 
blindness  in  different  parts  of  the  visual  field.  The  dimensions 
of  the  central  and  peripheral  display  conditions  were  chosen 
based  on  the  cortical  magnification  factor  (Rovamo  &  Virsu, 
1979)  such  that  the  volumes  of  primary  visual  cortex 
stimulated  by  either  the  central  or  peripheral  displays  were 
approximately  equal  and  one-half  the  volume  stimulated  by 
the  full-field  display.  This  choice  balanced  the  volume  of 
primary  visual  cortex  stimulated  by  the  central  and  peripheral 
displays. 

Observers  controlled  their  simulated  rotation  about  one  of 
two  axes  oriented  orthogonally  to  their  direction  of  motion:  a) 
the  vertical  (yaw)  axis,  or  b)  the  horizontal  (pitch)  axis. 
Because  the  displays  were  confined  to  a  90  x34  degree  (H  x 
V)  field  of  view,  the  manipulation  the  of  axis  of  rotational 
control  resulted  in  the  presentation  of  different  regions  of 
optical  flow  to  the  observer,  allowing  independent 
manipulation  of  visual  field  and  optical  flow  structure. 

Optical  flow  is  a  quantitative  description  of  the  dynamic 
pattern  of  light  projected  onto  a  point  of  observation,  such  as 
the  human  eye.  Information  carried  by  the  structure  of  optical 
flow  underlies  visual  guidance  of  locomotion  (Gibson,  1950; 
Gibson,  Olum,  &  Rosenblat,  1955).  For  example,  movement 
straight-ahead  through  the  environment  results  in  a  radial 
pattern  of  optical  flow  diverging  from  a  common  locus,  called 
the  focus  of  expansion  (FOE).  The  location  of  the  focus  of 
expansion  coincides  with  the  heading  direction.  In  principle, 
an  ideal  observer  could  determine  their  direction  of  heading  by 
finding  the  focus  of  expansion  in  optical  flow  through  a 
process  of  vector  triangulation  (W.H.  Warren,  1998). 

However,  heading  precision^  the  precision  with  which  the 
heading  direction  can  be  triangulated,  depends  on  which 
regions  of  optical  flow  are  visible  to  the  observer.  Formal 
analyses  of  optical  flow  structure  (Crowell  &  Banks,  1996; 
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Figure  3.  Plot  of  heading  precision  (see  text)  as  a  function  of  the 

horizontal  and  vertical  display  coordinates,  assuming  that 
the  observer  is  headed  toward  the  origin  (0,  0)  of  the 
displays.  The  shading  at  each  location  indicates  the 
precision  with  which  heading  can  be  recovered  by  an 
ideal  observer  based  on  an  analysis  of  image  element 
trajectories  at  that  location.  Note  that  a)  the  horizontal 
axis  represents  the  heading  axis  (see  text)  for  both  pitch- 
and  yaw-control,  and  b)  for  pitch-control  the  figure  is 
rotated  90  degrees  relative  to  the  actual  orientation  of  the 
display  used  in  the  experiment,  where  the  longer  axis 
was  horizontal  and  the  heading  axis  was  vertical. 

Dyre,  Morrow,  &  Richman,  2001)  indicate  that  the  regions  of 
optical  flow  with  the  worst  heading  precision  are  located  along 
the  axis  over  which  the  heading  direction  is  changing  (what 
will  subsequently  be  referred  to  as  the  heading  axis),  while 
regions  of  high  heading  precision  are  located  adjacent  to  the 
heading  direction  but  orthogonal  to  the  heading  axis  (see 
Figure  3).  To  illustrate,  imagine  the  task  of  driving  a  car. 
Automobile  driving  requires  that  we  control  our  heading 
direction  along  a  horizontal  (left-right)  axis,  so  the  most 
informative  regions  of  optical  flow  for  determining  our 
direction  of  heading  would  extend  vertically  above  and  below 
the  current  heading  direction. 

Ironically,  the  displays  used  in  previous  studies  of 
heading  performance  and  retinal  field  (Atchley  &  Andersen, 
1999;  Crowell  &  Banks,  1993;  Warren  &  Kurtz,  1992) 
presented  only  those  areas  of  optical  flow  falling  near  the 
heading  axis.  Regions  of  higher  heading  precision  located 
orthogonal  to  the  heading  axis  were  not  displayed.  To 
reproduce  a  similar  pattern  of  optical  flow  the  present  study 
used  a  yaw-control  task  where  the  heading  axis  passed 
horizontally  through  the  midline  of  a  90  degree  wide  display. 
Vertically,  the  display  subtended  only  34  degrees  of  visual 
angle,  so  informative  areas  of  optical  flow  above  and  below 
the  heading  axis  were  truncated  by  the  top  and  bottom  of  the 
display.  The  present  study  also  examined  heading 
performance  with  displays  that  included  these  areas  of  higher 
heading  precision  by  using  a  pitch-control  task.  For  pitch- 
control  the  heading  direction  varies  along  a  vertical  axis.  The 
more  informative  regions  of  optical  flow  located  orthogonal  to 


the  heading  axis  were  thus  visible  within  the  90  degree 
horizontal  extent  of  the  displays.  As  a  result,  the  pitch-control 
task  presented  regions  of  optical  flow  with  greater  heading 
precision  than  the  yaw-control  task,  regions  of  flow  that  have 
not  been  examined  in  previous  experiments  on  the  relative 
contribution  of  central  and  peripheral  vision  to  heading 
perception. 

The  primary  results  of  this  research  are  presented  in 
Figure  4,  which  shows  errors  in  heading  control  as  a  fiinction 
of  visual  field  (central,  peripheral,  or  full  field)  and  control 
axis  (yaw  vs.  pitch).  Results  for  the  yaw  control  task 
confirmed  the  results  of  previous  research  (Crowell  &  Banks, 
1993;  Warren  &  Kurtz,  1992):  the  central  visual  field  was 
necessary  and  sufficient  for  optimal  heading  performance. 

This  result  was  not  entirely  surprising  given  that,  like  these 
previous  studies,  the  informative  areas  of  optical  flow  above 
and  below  the  heading  axis  were  not  visible.  The  results  for 
pitch  control  show  a  remarkably  different  pattern:  equivalent 
heading  performance  for  central  and  peripheral  vision. 
Analysis  of  the  gaze  patterns  of  observers  suggest  that  this 
difference  in  performance  for  the  two  tasks  is  not  due  to 
differences  in  scanning  behavior.  In  both  tasks  observers 
generally  focused  their  gaze  at  or  near  the  center  of  the  display 
(see  Figure  5). 

What  are  the  implications  of  these  results  for  the  potential 
use  of  peripheral  displays  for  supporting  perception  of 
heading?  Most  importantly,  these  results  suggest  that 
observers  are  capable  of  accurate  control  of  their  heading 
direction  using  only  peripheral  vision,  but  only  if  critical 
regions  of  optical  flow  orthogonal  to  the  heading  axis  are 
visible.  Hence,  peripherally-located  displays  supporting 
heading  control  must  be  located  orthogonal  to  the  primary  axis 
of  heading  control.  For  piloting  a  surface  vehicle  where  the 
axis  of  heading  control  is  defined  by  the  horizon,  these 


Retinal  Field  of  View 


Figure  4.  Constant  and  variable  errors  for  pitch  and  yaw  control 
across  three  conditions  of  retinal  stimulation  (central, 
peripheral,  and  full;  see  text  for  details).  Error  bars 
represent  standard  errors  of  the  mean. 
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Control  Axis:  Yaw;  FOV:  Central;  Speed:  400ms'^ 
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Figure  5.  Typical  gaze  distributions  (fixation  dwell  times)  for  the 
yaw  (top)  and  pitch  (bottom)  control  tasks.  The  shading 
indicates  the  percentage  of  time  the  gaze  dwelled  within 
a  2  degree  radius  of  a  particular  display  coordinate. 

displays  would  ideally  be  located  above  and  below  the  horizon 
line.  For  piloting  flying  vehicles,  the  primary  axis  of  heading 
control  varies  depending  on  the  phase  of  flight.  Control  of 
glide-path  during  landing,  for  example,  requires  dual-axis 
control  of  pitch  and  yaw.  Displays  located  in  the  left  and  right 
peripheral  visual  fields  would  serve  control  of  pitch  quite 
effectively,  but  not  yaw.  Similarly,  displays  located  in  the 
upper  and  lower  peripheral  visual  fields  would  serve  control  of 
yaw  quite  effectively,  but  not  pitch. 

The  Role  of  Attention  in  Heading  Control  A  second  issue 
relevant  to  the  potential  of  peripheral  visual  displays  for 
controlling  heading  concerns  the  degree  of  attentional  or 
mental  workload  demanded  by  the  task  of  heading  control. 
Because  attention  is  a  mental  resource  with  limited  capacity, 
one  can  only  perceive  and  respond  to  a  limited  amount  of 
information  at  a  time.  The  potential  utility  of  peripheral 
displays  for  heading  depends  largely  on  their  ability  to  support 
accurate  heading  control  with  low  attentional  demand,  so  that 
maximal  mental  resources  are  available  for  pilots  to  devote  to 
other  tasks.  Past  research  indicates  that  attention  modulates 
perception  of  movement  (Cavanah,  1992;  Chaudhuri,  1990; 
Lankheet  &  Verstraten,  1995;  Rees,  Frith  &  Lavie,  1997). 

Previous  studies  of  attention  and  heading  judgments 
(Royden  &  Hildreth,  1999;  Wann,  Swapp  &  Rushton,  2000) 
produced  mixed  results  suggesting  that  attention  plays  little  or 
no  role  in  perceiving  heading.  However,  these  studies  did  not 
examine  continuous  manual  control  of  heading.  Instead,  they 
used  traditional  psychophysical  techniques  that  required 
observers  to  make  a  heading  judgment  after  viewing  a  very 
brief  display  of  optical  flow.  Further,  in  these  studies 
attentional  demand  was  manipulated  with  changes  in  the 
visual  information  present  in  the  display.  It  is  thus  difficult  to 
assess  whether  changes  in  heading  performance  were  due  to 
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changes  in  attentional  load,  memory  load  (because  responses 
were  delayed  until  after  the  displays  terminated),  or  simply 
due  to  changes  in  the  information  presented  in  the  displays.  In 
sum,  whcither  heading  control  is  adversely  affected  when 
attention  is  directed  to  other  information  or  tasks  remains 
unclear. 

The  purpose  of  the  present  study  was  to  more  precisely 
assess  the  attentional  demand  of  heading  control.  Observers 
continuously  controlled  heading  in  response  to  yaw 
disturbances  while  simultaneously  identifying  targets  in  a 
rapid  serial  visual  presentation  (RSVP)  of  arrays  of  numbers 
and  letters  (Schneider  &  Shiffrin,  1977).  This  dual-task 
method  differs  fi'om  previous  research  in  a  number  of 
important  ways.  First,  the  method  used  here  is  more  sensitive 
to  small  changes  in  attentional  demand  because  both  tasks 
required  continuous  monitoring  to  be  performed  well. 

Second,  performance  on  each  task  was  measured  while  the 
tasks  were  actually  being  performed,  rather  than  with  a 
delayed  response,  which  eliminates  post-display  memory 
demands  as  a  potential  confounding  factor.  Third,  heading 
performance  was  measured  in  the  context  of  continuous 
control,  rather  than  a  verbal  judgment,  which  more  accurately 
reflects  perception  of  heading  while  piloting  a  vehicle. 

Finally,  by  varying  target  mapping  and  target  set  size  to 
manipulate  attentional  demand  of  the  RSVP  task,  perceptual 
factors  were  held  constant — all  displays  consisted  of  alpha¬ 
numeric  RSVP  arrays  superimposed  over  the  optical  flow 
field. 

The  yaw-control  task  was  similar  to  the  full  visual  field 
condition  used  in  the  study  of  field-of-view  and  heading 
control  discussed  previously.  The  heading  displays  simulated 
linear  translation  through  a  star  field  combined  with  a  sum-of- 
sines  disturbance  about  the  yaw  axis.  Over  a  60  s  period, 
observers  controlled  yaw  to  maintain  their  perceived  direction 
of  heading  straight  ahead  toward  the  center  of  the  displays. 

For  the  search  task,  the  RSVP  arrays  were  superimposed  over 
the  optical  flow  pattern  and  yoked  to  the  observer’s  gaze 
direction  in  real-time  using  an  eye-head  tracking  system  (see 
Figure  6).  This  allowed  observers  to  look  wherever  they 


Horizontal  Display  Coordinate 


Figure  6.  Pictorial  representation  of  the  optical  flow  fields  used  to 
examine  the  influence  of  attention  on  heading 
performance.  Each  arrow  represents  the  velocity  of  a 
moving  dot.  The  array  of  letters  was  centered  on  the 
gaze  direction  and  moved  with  it  across  the  displays. 
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needed  to  best  perform  the  heading  control  task,  while 
maintaining  the  visibility  of  the  RSVP  task.  The  RSVP  arrays 
consisted  of  four  letters  or  numbers  that  changed  identities 
every  200  ms,  spaced  equally  at  the  comers  of  a  virtual  square 
centered  on  the  gaze  direction.  Prior  to  the  onset  of  each 
RSVP  sequence,  a  set  of  either  one  or  four  targets  was 
presented  to  the  observer.  If  any  of  these  targets  was  detected 
in  the  subsequent  RSVP  sequence,  observers  responded  by 
pressing  a  button  on  the  same  joystick  used  to  control  heading. 
Observers  were  instmcted  to  attend  and  respond  to  a)  the 
heading  display  only  (single-task  heading),  the  search  display 
only  (single-task  search),  or  both  displays  (dual  task). 

To  manipulate  the  attentional  demand  of  the  RSVP  task, 
two  target  mappings  were  used:  a)  consistent  mapping  or  CM, 
where  target  and  distracter  items  were  selected  from  unique 
sets  (e.g.,  letter  targets  among  digit  distracters),  and  b)  varied 
mapping  or  FM,  where  target  and  distracter  items  were 
selected  from  the  same  set  (targets  for  a  given  trial  could  serve 
as  distracters  on  other  trials).  Target  set  size  was  also 


Figure  7.  Search  Task  Errors  as  a  function  of  type  of  mapping, 

consistent  (CM)  vs.  variable  (VM),  and  set  size  (1  or  4). 
These  data  are  averaged  across  all  other  experimental 
factors,  which  showed  no  reliable  effects.  Error  bars 
represent  standard  errors  of  the  mean. 

assumed  to  manipulate  attentional  demand  since  greater  set 
sizes  would  place  greater  demands  on  working  memory. 
Schneider  and  Shffirin  (1977)  showed  that  with  training,  target 
set  size  does  not  influence  target  detection  accuracy  in  the  CM 
search,  which  demands  little  attention  and  can  be  carried  out 
in  parallel.  In  contrast,  even  after  extensive  training,  target 
detection  errors  increase  as  set  size  increases  for  VM  search, 
which  demands  greater  attention  and  requires  a  serial  search  of 
the  RSVP  array.  Note  that  the  CM  and  VM  search  conditions 
both  require  search  of  an  RSVP  sequence,  thus  the  visual 
conditions  were  equated  for  conditions  of  low  (CM)  and  high 
(VM)  attentional  demand. 

If  heading  perception  demands  attentional  resources,  one 
of  two  outcomes  could  occur  depending  on  the  priority  given 
to  the  heading  and  search  tasks  by  participants.  If  the  search 
task  receives  higher  priority,  then  attention  to  the  heading- 
control  task  will  be  reduced  and  heading  errors  will  increase. 


particularly  when  the  heading  task  is  paired  with  the  more 
attentionally-demanding  VM  search  task.  If,  on  the  other 
hand,  the  heading-control  task  receives  higher  priority, 
heading  performance  might  remain  constant  and  the  effect  of 
increased  attentional  demand  for  the  dual  task  will  be  evident 
in  an  interaction  between  task  (search  alone  vs.  dual  task)  and 
target  mapping  (VM  vs.  CM)  on  search  errors,  with  greater 
single-to-dual-task  increases  in  error  for  the  more 
attentionally-demanding  VM  search  task  as  compared  to  the 
lower  attentionally-demanding  CM  search  task.  In  addition  to 
these  performance  measures,  observers  completed  the  NASA 
Task-Load  Index  (TLX)  questionnaire  for  each  unique 
combination  of  search  mapping,  search  set  size,  and  task 
(single  vs.  dual)  so  that  the  subjective  mental  workload 
demanded  by  the  various  display  conditions  could  be  assessed. 

In  general,  the  RSVP  search  results  for  both  single-  and 
dual-task  conditions  showed  the  typical  pattern  found  in 
previous  research.  The  number  of  errors  was  greater  for  VM 
as  compared  to  CM  search,  particularly  for  set  size  4  (see 


single  dual 
Task 

Figure  8.  Constant  heading  errors  as  a  ftinction  of  type  of  task, 
single  vs,  dual,  and  mapping,  consistent  (CM)  vs. 
variable  (VM).  Error  bars  represent  standard  errors  of 
the  mean. 


Figure  7).  These  results  confirm  that  VM  search  and  larger  set 
sizes  demand  more  attention  than  CM  search.  Further,  the 
consistency  of  this  pattern  across  single-  and  dual-task 
conditions  suggests  that  participants  did  not  reduce  the 
attentional  resources  allocated  to  the  search  task  when  it  was 
paired  with  the  heading  control  task.  This  result  is  not 
surprising  given  that  explicit  feedback  was  provided  for  search 
performance  but  not  heading  performance.  The  subjective 
workload  ratings  calculated  from  the  NASA-TLX  were 
consistent  with  the  search  data:  VM  search  produced  higher 
subjective  workload  than  CM  search,  particularly  for  set  size 
4. 

Participants  also  reported  higher  subjective  workload  on 
the  NASA-TLX  when  the  heading  and  search  tasks  were 
paired  together  as  compared  to  performing  either  task 
independently.  This  result  suggests  that  participants  perceived 
both  tasks  to  demand  mental  resources.  Performance  on  the 
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heading  control  task  was  consistent  with  this  perception.  As 
illustrated  in  Figure  8,  constant  errors  in  heading  control  were 
increased  by  approximately  40%  when  the  heading  task  was 
paired  with  the  VM  search  task  relative  to  the  heading  task 
alone  or  when  it  was  paired  with  the  CM  search  task.  This 
increase  in  heading  error  for  only  VM  searches  suggests  that 
participants  shifted  attentional  resources  from  the  heading 
control  task  to  the  attentionally-demanding  VM  search  task  to 
maintain  search  performance. 

This  result  conclusively  demonstrates  that  visual  control 
of  heading  from  optical  flow  demands  at  least  some  attentional 
resources,  and  by  extension,  pilots  would  need  to  allocate 
attentional  resources  to  process  virtual  displays  designed  to 
support  heading  control.  However,  the  amount  of  attentional 
resources  demanded  by  the  heading  task  appears  to  be 
relatively  modest,  as  the  average  increase  in  heading  error  for 
the  VM  search  condition  was  only  0.4  degrees.  Whether  this 
small  increase  in  error  is  operationally  significant  is  difficult 
to  assess,  but  it  seems  reasonable  to  expect  that  inattention  to 
the  alpha-numeric  display  symbology  currently  used  to  specify 
heading  would  result  in  a  much  greater  degree  of  error. 

Further  research  is  needed  to  empirically  test  this  hypothesis. 
However,  the  current  results  strongly  suggest  that  virtual 
displays  using  optical  flow  may  convey  heading  information 
in  a  manner  that  requires  only  a  very  small  amount  of 
attentional  resources.  Indeed,  the  attentional  effects  on 
heading  performance  found  here  were  only  evident  due  to  the 
strong  attentional  demand  of  the  VM  search  and  high 
sensitivity  of  the  measure  of  heading  performance.  Previous 
research  with  less  attentionally-demanding  tasks  and  less 
sensitive  measures  failed  to  find  that  heading  performance  was 
attention-limited  (Royden  &  Hildrith,  1999). 

In  conclusion,  the  current  research  demonstrates  that 
virtual  displays  using  optical  flow  for  specifying  heading 
demand  relatively  little  attention  and  may  be  especially  useful 
for  supporting  precise  heading  control  under  conditions  of 
high  mental  workload,  such  as  landings.  However,  further 
research  is  needed  to  determine  more  precisely  the  potential  of 


Figure  9.  Pictorial  representation  of  the  optical  flow  fields  used  to 
examine  the  influence  of  motion  transparency  on 
heading  performance.  Each  arrow  represents  the 
velocity  of  a  moving  dot.  The  crosshairs  indicate  the 
heading  directions  specified  by  the  two  transparently 
overlaid  flow  fields.  These  crosshairs  were  not  included 
in  the  displays  used  in  the  experiments. 


virtual  displays  for  such  tasks. 

Motion  Transparency,  Attention,  and  Heading 
Perception,  There  are,  however,  some  potential  drawbacks  in 
transparently  superimposing  a  virtual  HUD  containing  optical 
flow  over  a  pilot’s  view  of  the  environment.  For  example, 
studies  of  heading  perception  in  the  context  of  moving 
transparent  objects  (Warren  &  Saunders,  1995;  Royden  & 
Hildreth,  1996)  found  that  moving  display  elements  specifying 
heading  may  be  perceptually  integrated  with  the  motion  of 
objects,  and  this  integration  can  result  in  inaccuracies  in 
heading  perception,  particularly  if  the  optical  flows  of  the 
moving  object  and  background  specify  different  heading 
directions. 

To  further  examine  this  phenomenon  a  series  of 
experiments  was  conducted  that  examined  heading  judgments 
and  control  in  the  context  of  transparently  superimposed 
optical  flow,  such  as  that  produced  by  using  a  HUD  to  overlay 
a  virtual  optical  flow  display  on  the  optical  flow  directly 
projected  from  the  environment  through  the  windscreen.  The 
purpose  of  this  research  was  to  determine  the  effect  on 
heading  accuracy  of  misalignments  between  the  optical  flow 
presented  in  a  HUD  and  that  projected  by  real-world  objects. 
Misalignments  like  these  are  common,  for  example,  in  night- 
warfare  settings  where  infrared  sensor  imagery  is  projected 
onto  a  HUD  or  HMD.  Generally,  the  sensor  producing  the 
imagery  is  not  aligned  with  the  pilot’s  eye,  so  the  optical  flow 
of  the  sensor  imagery  is  not  exactly  aligned  with  the  optical 
flow  of  the  environment  viewed  directly  through  the 
windscreen.  This  misalignment  could  result  in  misperception 
of  the  direction  of  heading  and  compromise  safe  and  effective 
flight  control. 

The  general  method  used  in  these  experiments  was  to 
present  participants  with  simulations  of  observer  movement 
through  two  overlapping,  but  independently  moving,  star- 
fields  similar  to  those  used  in  the  experiments  previously 
described,  except  that  each  flow  field  specified  a  unique 
heading  direction  (see  Figure  9).  Participants  judged  how 
many  distinct  heading  directions  they  perceived  and  estimated 
the  heading  direction  specified  by  one  of  the  two  flow  fields. 
The  effects  of  motion  transparency  on  heading  performance 
were  assessed  by  examining  the  errors  in  these  judgments  as  a 
function  of  angular  separation  between  the  heading  directions. 

The  direction  of  errors  in  heading  perception  were  coded 
as  either  attraction  errors — ^the  judged  heading  direction  lay 
between  the  two  actual  heading  directions — or  repulsion 
errors — the  judged  heading  direction  lay  outside  the  area 
between  the  two  actual  heading  directions,  in  other  words,  the 
perceived  heading  direction  of  the  judged  flow  field  was 
repulsed  away  from  the  non-judged  flow  field.  Typical  results 
are  shown  in  Figure  10.  Note  that  significant  errors  in 
perception  of  heading  occur  in  response  to  misaligned 
transparent  optical  flow.  For  small  misalignments,  observers 
perceive  the  two  flow  fields  as  one  coherent  flow  field  and 
errors  of  attraction  occur  that  suggest  the  visual  system  is 
averaging  the  flow  vectors  to  estimate  the  heading  direction. 
However,  as  angular  separation  increases,  observers  more 
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Figure  10.  Number  of  flow  fields  perceived  (top)  and  errors  in 
perceived  heading  direction  (bottom)  as  a  function  of 
angular  separation  between  the  heading  directions 
simulated  by  the  two  flow  fields.  Error  bars  represent 
standard  errors  of  the  mean. 


likely  perceive  two  distinct  flow  fields  (motion  transparency) 
and  repulsion  errors  generally  increase. 

This  pattern  of  errors  is  consistent  with  a  motion  contrast 
effect  (Marshak  &  Sekuler,  1979)  that  would  result  if  neural 
units  tuned  to  detect  heading  in  different  directions,  such  as 
those  proposed  by  Perrone  and  Stone  (1994,  1998),  mutually 
inhibit  one  another.  Such  inhibitory  interactions  between 
neural  units  are  common  in  the  visual  system,  affecting,  for 
example,  perception  of  orientation  and  linear  motion  direction 
(Marshak  &  Sekuler,  1979).  Hence,  it  is  reasonable  to  expect 
that  higher  level  cortical  motion  mechanisms  involved  in 
heading  perception  may  also  exhibit  mutual  inhibition. 

An  alternative  explanation  for  the  repulsion  errors  is  that 
they  result  from  induced  motion’,  the  illusory  perception  of 
foreground  motion  resulting  from  movement  of  the 
background.  The  repulsion  errors  found  in  this  experiment  are 
consistent  with  the  errors  predicted  by  induced  motion  of  the 
judged  flow-field  (perceived  as  foreground)  relative  to  the 
non-judged  flow  field  (perceived  as  background).  Hence,  it  is 
unclear  whether  the  repulsion  errors  observed  in  these 
experiments  result  from  motion  contrast  or  induced  motion. 
Regardless  of  the  underlying  cause,  the  results  indicate  that 
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Figure  1 1 .  Number  of  flow  fields  perceived  (top)  and  errors  in 
perceived  heading  direction  (bottom)  as  a  function  of 
angular  separation  between  the  heading  directions 
simulated  by  the  two  flow  fields  and  color  coding  of  the 
dots  in  each  flow  field.  Error  bars  represent  standard 
errors  of  the  mean. 


operationally  significant  heading  errors  can  be  induced  by 
misaligned  transparent  motion  of  two  optical  flow  fields 
composed  of  homogenous  elements. 

Further  experiments  examined  whether  pre-attentive  and 
attentional  selection  of  flow  components  based  on  a  non¬ 
motion  attribute,  color,  modulates  the  heading  errors 
associated  with  transparent  flow.  This  could  happen,  for 
example,  when  a  pilot  attentively  selects  the  uniquely-colored 
optical  flow  presented  within  a  virtual  HUD  for  processing 
and  ignores  the  optical  flow  projected  from  the  real-world 
environment  through  the  windscreen.  This  attentional 
selection  could,  in  principle,  reduce  or  eliminate  errors  in 
heading  perception  resulting  from  misaligned  flow  by 
reducing  the  saliency  of  optical  flow  information  in  the  non- 
judged  flow  field.  Alternatively,  attentional  selection  of  one 
flow  component  could  increase  errors  in  heading  perception 
by  enhancing  segregation  of  foreground  and  background,  and 
hence,  an  induced  motion  effect. 

To  test  these  competing  hypotheses,  further  motion 
transparency  experiments  were  conducted  with  the  two  flow 
components  coded  in  distinct  colors.  Figure  1 1  shows  heading 
errors  with  these  displays  (labeled  “separated  red”  and 
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Figure  12.  Number  of  flow  fields  perceived  (top)  and  errors  in 
perceived  heading  direction  (bottom)  as  a  function  of 
angular  separation  between  the  heading  directions 
simulated  by  the  two  flow  fields,  color  coding  of  the 
dots,  and  attentional  instructions.  Error  bars  represent 
standard  errors  of  the  mean. 

“separated  green”)  as  compared  to  either  homogeneously- 
colored  displays  (labeled  “all  green”,  “all  red”,  or  “all  white”), 
or  displays  where  all  colors  were  equally  integrated  across 
both  flow  components  (“integrated”).  For  these  data, 
participants  were  unaware  of  the  color-coding  manipulation 
and  no  attentional  instructions  were  given.  Therefore,  any 
effect  of  color  coding  was  presumed  to  result  from  pre- 
attentive  processes.  As  can  be  seen  in  Figure  1 1,  no  reliable 
effects  of  color  coding  were  found,  which  suggests  that  simply 
distinguishing  optical  flow  components  by  color  does  not  alter 
the  effects  of  motion  transparency. 

However,  color  coding  was  found  to  affect  heading  errors 
resulting  from  motion  transparency  when  observers  were 
explicitly  instructed  to  attend  only  display  elements  of  a 
particular  color.  Figure  12  shows  the  results  of  an  experiment 
with  color-coded  flow  fields  where  participants  were  explicitly 
instructed  to  attend  “all  the  dots,”  “only  the  red  dots,”  or  “only 
the  green  dots.”  The  magnitude  of  repulsion  errors  increased 
when  observers  attended  one  flow  field  and  ignored  the  other 
(the  lines  labeled  “attend  red”  or  “attend  green”  in  Figure  12). 
These  results  are  consistent  with  attentional  enhancement  of 
induced  motion.  An  important  practical  implication  of  these 
results  is  that  color  coding  optical  flow  in  virtual  HUDs  will 
not  reduce  heading  errors  that  result  from  motion 
transparency,  and  may  actually  increase  these  errors,  if  pilots 


are  instructed  to  attentionally  select  the  moving  HUD 
symbology. 

Summary  and  Conclusions  of  Research  on  Heading 
Perception  and  Control  Several  practical  conclusions  can  be 
drawn  from  this  body  of  research  on  heading  perception. 

First,  this  research  suggests  that  the  heading  direction  can  be 
accurately  perceived  from  optical  flow  processed  in  the 
peripheral  visual  fields  if  informative  regions  of  optical  flow 
structure  are  visible.  These  informative  areas  of  optical  flow 
lie  in  directions  orthogonal  to  the  heading  axis,  adjacent  to  the 
current  heading  direction,  which  suggests  that  for  tasks  such 
as  yaw-control,  where  the  heading  axis  is  horizontal, 
peripheral  visual  displays  should  be  located  directly  above  or 
below  the  most-likely  direction  of  heading.  Moreover,  the 
research  on  attentional  demands  of  heading  perception 
suggests  that  errors  in  heading  control  do  not  increase 
dramatically  even  when  attention  is  directed  to  a  demanding 
secondary  task.  Together,  these  results  indicate  that  heading 
control  during  periods  of  high  mental  workload  could 
potentially  benefit  from  peripherally-located  virtual  optical 
flow  displays.  One  important  caveat,  however,  comes  from 
the  research  on  motion  transparency  and  heading  perception: 
Overlaying  optical  flow  from  a  HUD  on  the  scene  viewed 
through  the  windscreen  can  produce  systematic  errors  in 
heading  perception  if  the  heading  directions  specified  by  each 
optical  flow  pattern  are  significantly  misaligned. 

Research  on  Egospeed  Perception  and  Control 

The  importance  of  accurate  perception  of  the  speed  of 
self-motion,  or  egospeed,  for  successful  vehicular  control  is 
evidenced  by  the  prominence  of  airspeed  indicators  in  aviation 
cockpits  and  speedometers  on  automobile  dashboards.  These 
highly-symbolic  displays  of  speed  information  augment  the 
optical  flow  visible  through  the  windscreen  and  significantly 
increase  precision  in  speed  control,  particularly  when  fog  or 
other  optical  conditions  might  produce  speed  illusions 
(Snowden,  Stimpson,  &  Ruddle,  1998).  However,  the  symbols 
used  in  airspeed  indicators  and  speedometers  require  direct 
visual  fixation,  visual  attention,  and  other  cognitive  processes 
to  perceive  and  interpret — resources  that  might  be  scarce 
during  difficult  aviation  operations  such  as  landing  or  threat- 
evasion.  Controlling  speed  under  these  circumstances  might 
be  enhanced  by  speed  information  displayed  as  optical  flow  in 
a  virtual  HUD,  a  speed  indicator  that  could  potentially  be 
processed  pre-attentively  by  the  visual  periphery  with  less 
cognitive  intervention,  thereby  freeing  mental  resources  and 
reducing  workload.  To  gain  an  understanding  of  the  potential 
of  such  displays  and  how  they  might  best  be  designed,  it  is 
first  necessary  to  understand  how  egospeed  is  visually 
perceived  and  controlled. 

The  research  summarized  below  examined  three  issues: 

1)  the  influence  of  texture  density  on  perceived  egospeed  and 
whether  this  influence  varies  as  a  function  of  visual  field, 

2)  the  influence  of  texture  density  on  egospeed  control,  and 

3)  the  misperception  of  changes  in  egospeed  from  changes  in 
image  speed  variability  due  to  motion  parallax.  For  more 
detailed  information  please  refer  to  the  theses  and  other 
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publications  listed  in  the  section  titled,  “Publications 
Stemming  from  the  Research  Effort.” 

Texture  Density  Effects  on  Egospeed  Perception.  Owen, 
Wolpert,  and  Warren  (1984)  identified  discontinuity  rate  and 
global  optical  flow  rate  as  two  sources  of  information  that 
specify  egospeed.  Discontinuity  rate,  which  is  determined  by 
counting  the  number  of  texture  elements  passing  a  fixed  visual 
reference  per  unit  time  (temporal  frequency),  is  dependent  on 
egospeed  and  texture  density,  and  is  generally  expressed  in 
units  of  discontinuities/s  or  Hz  .  Global  optical  flow  rate  is  the 
rate  of  egospeed  scaled  in  altitude  units.  It  is  generally 
expressed  in  units  such  as  eyeheights/s,  where  one  eyeheight 
equals  the  altitude  of  the  observer’s  eye  above  a  ground  plane. 
Usually,  these  two  information  sources  redundantly  specify 
egospeed;  however,  sometimes  they  can  conflict.  For  example, 
global  optical  flow  rate  will  change  if  an  observer  changes 
altitude,  and  discontinuity  rate  will  change  with  variations  in 
texture  density,  such  as  the  increase  in  the  density  of  flora 
while  approaching  a  river  passing  through  a  desert. 

The  majority  of  egospeed  research  has  shown  that  while 
both  discontinuity  rate  and  global  optical  flow  rate  contribute 
to  the  perception  of  egospeed,  discontinuity  rate  dominates 
(Denton,  1980;  Owen  et  al.,  1984),  perhaps  reflecting  the 
involvement  of  cognitive  processes  such  as  edge  counting 
(Larish  &  Flach,  1990).  Other  research  (Dyre,  1997)  found 
the  exact  opposite  result:  acceleration  judgments  were 
dominated  by  global  optical  flow  rate.  Indeed,  its  effect  was 
72  times  stronger  than  the  effect  of  discontinuity  rate,  which 
suggests  that  effects  of  discontinuity  rate  on  egospeed  might 
reflect  the  effects  of  temporal  frequency  on  motion  sensing 
(Watamaniuk,  Grzywacz,  &  Yuille,  1993),  A  number  of 


Global  Optical  Flow  Rate  (eyeheights/s) 

Figure  13.  Normalized  mean  image  flow  rate  and  egospeed 

estimates  plotted  as  a  function  of  global  optical  flow  rate 
and  discontinuity  rate.  Panel  (a)  shows  normalized 
image  flow  rates  calculated  for  displays  replicating  those 
of  Larish  &  Flach  (1990).  Note  that  mean  image  flow 
rate  is  more  correlated  with  discontinuity  rate  than  global 
optical  flow  rate.  Panel  (b)  shows  normalized  image 
flow  rates  calculated  for  displays  where  mean  image 
flow  rate  was  more  correlated  with  global  optical  flow 
rate.  Panels  (c)  and  (d)  show  normalized  egospeed 
estimates  for  these  displays. 


methodological  differences  between  these  studies  may  account 
for  these  conflicting  results.  The  present  study  aimed  to 
determine  which  factors  produced  these  contradictory  results, 
and  identify  the  most  relevant  information  for  perceiving 
egospeed.  To  this  end,  experiments  were  conducted  with  a 
variety  of  displays  and  experimental  tasks  that  bridged  the 
differences  between  these  previous  studies,  examining  issues 
such  as  frame-update  rate,  field  of  view,  and  optical  flow 
structure.  The  overall  results  of  these  experiments  suggested 
that  one  methodological  factor  in  particular  appeared  to  best 
account  for  the  conflicting  results:  covariation  of  mean  image 
flow  rate  with  the  experimental  manipulations  of  discontinuity 
rate  or  global  optical  flow  rate. 

Mean  image  flow  rate  is  the  average  angular  rate  at  which 
the  images  corresponding  to  environmental  objects  move  in 
optical  flow  projected  onto  an  image  surface.  Because  mean 
image  flow  rate  is  a  description  of  optical  speed,  it  is  closely 
related  to  global  optical  flow  rate,  although  it  is  expressed  in 
angular  units  of  degrees  per  second  rather  than  environmental 
units  such  as  eyeheights/s.  As  a  result,  mean  image  flow  rate 
and  global  optical  flow  rate  are  highly  correlated.  In  contrast, 
mean  image  flow  rate  and  discontinuity  rate  are  conceptually 
distinct.  The  former  is  a  description  of  speed,  the  latter  a 
description  of  temporal  frequency.  For  regular  or  uniformly 
random  distributions  of  texture  in  the  environment, 
discontinuity  rate  and  mean  image  flow  rate  are  independent. 

Unfortunately,  the  manipulations  of  discontinuity  rate 
used  in  previous  studies  (Denton,  1980;  Larish  &  Flach,  1990; 
Owen  et  al.,  1984)  resulted  in  large,  statistically  reliable, 
correlations  between  discontinuity  rate  and  mean  image  flow 
rate,  which  may  in  part  explain  the  relatively  large 
discontinuity  rate  effects  found.  For  example,  consider  the 
data  presented  in  Figure  13.  This  figure  shows  the 
relationship  between  mean  image  flow  rate  and  egospeed 
estimates  for  displays  simulating  straight-ahead  observer 
movement  over  a  groundplane,  where  global  optical  flow  rate 
and  discontinuity  rate  were  manipulated  by  varying  altitude 
and  texture  density,  respectively.  The  left  column  of  panels 
corresponds  to  displays  similar  to  those  used  by  Larish  and 
Flach  (1990).  For  these  displays,  mean  image  flow  rate  was 
more  correlated  with  discontinuity  rate  than  global  optical 
flow  rate  and  discontinuity  rate  dominated  egospeed  estimates. 
The  right  column  of  panels  corresponds  to  displays  where 
mean  image  flow  rate  was  more  correlated  with  global  optical 
flow  rate  than  discontinuity  rate.  For  these  displays,  global 
optical  flow  rate  dominated  egospeed  estimates.  Geometric 
analysis  of  the  displays  used  by  Denton  (1980),  Dyre  (1997), 
Larish  and  Flach  (1990),  and  Owen  et  al.  (1984)  revealed  the 
same  pattern:  the  cue  that  dominated  perceived  egospeed  was 
always  more  strongly  correlated  with  mean  image  flow  rate. 

Although  these  results  suggest  that  mean  image  speed 
rather  than  discontinuity  rate  per  se  dominates  perceived 
egospeed,  they  do  not  completely  rule  out  any  influence  of 
texture  density  and  temporal  frequency.  Indeed, 
manipulations  of  texture  density  and  temporal  frequency 
independent  of  mean  image  speed  have  been  shown  to 
influence  the  perceived  speed  of  2D  linear  object  motion 
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Figure  14.  Ratio  of  perceived  speed  to  standard  speed  plotted  as  a 
function  of  the  ratio  of  the  comparison  dot  density  to  the 
standard  dot  density.  The  solid  circles  represent 
standardized  perceived  speeds  calculated  from  the 
proportion  of  “faster”  judgments  for  each  stimulus. 

Error  bars  represent  standard  errors  of  the  mean.  The 
solid  line  represents  the  best  fit  power  function  of  the 
form 

perceived  speed  _  (  comparison  dot  density  |  > 
standard  speed  standard  dot  density  J 
where  the  exponent  (Z?  =  0.123)  is  the  slope  of  the  linear 
increase  (in  log-log  space)  of  perceived  speed  with 
increasing  dot  density.  The  dotted  lines  represent  95% 
confidence  intervals  for  the  best-fit  power  function 
described  above.  The  dashed  line  represents  the 
predicted  perceived  speed  based  on  the  exponent  (b  = 

0.1 1 8)  found  by  Watamaniuk  et  al.  (1993)  for  2-D 
frontal  parallel  motion. 

(Watamaniuk  et  al,  1993)  and  Dyre  (1997)  found  weak  but 
reliable  texture  density  effects  on  the  perception  of 
accelerating  egospeed.  To  further  examine  this  issue,  the 
present  study  varied  texture  density  independently  of  mean 
image  speed  using  displays  that  manipulated  discontinuity  rate 
and  global  optical  flow  rate  in  a  manner  similar  to  Dyre  (1997) 
and  measured  their  effects  on  perceived  speed  using  a 
standard-comparison  discrimination  technique  similar  to 
Watamaniuk  et  al.  (1993).  The  results  are  presented  in  Figure 
14.  Variations  in  texture  density  produced  small  but 
statistically  reliable  effects  on  perceived  egospeed  (e.g.,  a 
1000%  increase  in  density  resulted  in  a  10%  increase  in 
perceived  egospeed).  The  magnitude  of  the  texture  density 
effect  on  perceived  egospeed  was  strikingly  similar  to  the 
magnitude  of  the  texture  density  effect  on  the  perceived  speed 
of  2D  linear  object  motion  found  by  Watamaniuk  et  al.  (1993). 
This  result  suggests  that  texture  density  most  likely  affects 
representations  of  speed  early  in  the  2D  motion  sensing 
process  and  this  effect  filters  through  to  higher-level  3D 


motion  processing  underlying  phenomenon  such  as  perception 
of  egospeed. 

This  conclusion  is  confirmed  by  further  experiments 
conducted  to  assess  how  the  influence  of  discontinuity  rate 
and  global  optical  flow  rate  may  change  as  a  function  of  field- 
of-view.  Similar  to  the  research  on  visual  field  and  heading 
described  above,  visual  field  was  manipulated  by  yoking  a 
moving  mask  (or  aperture)  to  the  gaze  direction  measured  in 
real-time  within  a  large  90  x  34  deg.  (H  x  V)  field  of  view 
display  by  an  eye-head  tracking  apparatus.  Seven  different 
fields  of  view  were  examined,  including  small  circular 
apertures  (7  deg.  radius)  centered  on  the  gaze  direction,  large 
field  peripheral  displays  spanning  30-40  degrees,  and  the  full 
90  X  34  deg.  field  of  view.  No  matter  the  visual  field 
condition,  both  discontinuity  rate  and  global  optical  flow  rate 
had  consistently-sized  effects  on  egospeed  estimates.  There 
was  no  evidence  suggesting  that  the  mechanism  by  which 
texture  density  affects  egospeed  perception  varies  as  a 
function  of  visual  field,  an  observation  that  is  easier  to 
reconcile  with  the  hypothesis  that  texture  density  effects  on 
speed  occur  during  early  motion  sensing,  rather  than  being 
part  of  higher-level  cognitive  processing. 

Texture  Density  and  Control  of  Speed,  Experiments  were 
also  conducted  to  examine  the  influence  of  discontinuity  rate 
and  global  optical  flow  rate  on  the  control  of  egospeed. 
Previous  studies  all  measured  open-loop  magnitude 
estimations  or  discriminations  of  perceived  egospeed. 
However,  discontinuity  rate  might  have  less  of  a  biasing  effect 
on  perception  of  egospeed  when  a  pilot  is  engaged  in  closed- 
loop  control,  where  continuous  visual-motor  feedback  might 
serve  to  disambiguate  true  changes  in  egospeed  fi'om  the 
illusory  effects  of  discontinuity  rate.  Similar  to  the 
experiments  on  control  of  yaw  discussed  previously  in  this 
report,  observers  were  shown  displays  simulating  straight¬ 
ahead  movement  over  a  groundplane  defined  by  dots.  The 
displays  lasted  40  s  and  filled  a  field  of  view  spanning  90  x  34 
deg.  (H  x  V).  Observers  attempted  to  maintain  a  constant 
speed  in  response  to  a  simulated  head-wind  disturbance 
defined  by  a  sum-of-sines  function.  Independent  changes  in 
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Figure  15.  Mean  errors  in  egospeed  control  as  functions  of  global 
optical  flow  rate  and  discontinuity  rate.  Error  bars 
represent  standard  errors  of  the  mean. 
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discontinuity  rate  and  global  optical  flow  rate  were  introduced 
by  exponentially  increasing  or  decreasing  the  density  of 
visible  dots,  or  the  simulated  altitude  over  the  plane, 
respectively.  The  mean  errors  in  speed  control  computed  for 
each  combination  of  discontinuity  rate  and  global  optical  flow 
rate  are  shown  in  Figure  15.  Two  results  in  particular  are  of 
note. 

First,  discontinuity  rate  had  no  reliable  effect  on  control 
of  egospeed,  while  global  optical  flow  rate  had  a  large, 
reliable  effect.  This  result  is  consistent  with  the  relatively 
small  effect  of  discontinuity  rate  found  by  the  experiments 
described  above  that  measured  speed  estimations  or 
discriminations,  but  directly  contradicts  previous  research  that 
found  egospeed  control  to  be  profoundly  influenced  by 
discontinuity  rate  (Denton,  1980).  Again,  the  critical  factor 
that  explains  the  difference  in  results  is  whether  discontinuity 
rate  is  correlated  with  changes  in  mean  image  speed,  as  in 
Denton’s  study,  or,  as  in  this  study,  discontinuity  rate  was 
uncorrelated  with  mean  image  speed.  Mean  image  speed 
appears  to  be  the  critical  factor  not  only  for  visually 
perceiving  egospeed,  but  also  for  controlling  egospeed. 

Second,  participants  tended  to  increase  speed  (positive 
errors)  over  the  trial  duration  for  almost  every  type  of  display 
examined,  including  those  in  which  global  optical  flow  rate 
and  discontinuity  rate  were  perfectly  correlated  with  changes 
in  egospeed.  The  greatest  increases  in  error  occurred  for 
displays  in  which  altitude  increased  exponentially  resulting  in 
a  concomitant  decrease  in  global  optical  flow  rate.  The 
decreasing  optical  flow  rate  for  these  displays  most  likely 
resulted  in  the  perception  of  slower  egospeed  and  observers 
compensated  by  speeding  up.  In  comparison,  little  or  no  error 
occurred  for  displays  in  which  altitude  decreased,  causing 
global  optical  flow  rate  to  increase.  The  relatively  constant 
speed  maintained  in  these  conditions  suggests  that  observers 
perceived  speed  to  be  unchanging  despite  the  increase  in 
global  optical  flow  rate. 

An  additional  experiment  that  examined  the  effect  of 
texture  density  on  the  precision  of  speed  control  replicated  this 
“speeding-up”  phenomenon.  Unlike  the  experiments 
discussed  previously,  which  examined  how  variations  in 
texture  density  might  bias  estimates  and  control  of  egospeed, 
this  experiment  attempted  to  determine  which  constant  level 
of  display  density  provided  optimal  speed  control 
performance.  It  used  methods  and  displays  similar  to  those 
used  in  the  experiments  on  speed  control  described  previously, 
except  that  a)  texture  density  and  altitude  were  constant  for  the 
duration  of  the  display  and  b)  the  number  of  display  elements 
(texture  density)  was  varied  across  displays.  Observers 
attempted  to  maintain  constant  speed  in  response  to  a 
simulated  headwind  disturbance  defined  by  a  sum-of-sines 
function.  Like  the  previously  described  study  of  egospeed 
control,  control  of  egospeed  was  actually  quite  poor;  observers 
exhibited  positive  speed  errors  for  all  the  display  densities 
examined  (i.e.,  they  sped  up  over  the  course  of  each  trial), 
although  the  increase  in  speed  was  minimized  at  moderate 
display  densities  (see  Figure  16). 


Density  (dots/deg^) 


Figure  16.  Mean  errors  in  egospeed  control  as  functions  of  texture 
density  and  target  egospeed.  Error  bars  represent 
standard  errors  of  the  mean. 

How  can  this  consistent  increase  in  egospeed  be 
explained?  One  possibility  is  that  observers  sped  up  to 
compensate  for  perceptual  adaptation  to  the  optical  flow  over 
time,  which  could  lead  to  decreased  motion  sensitivity  and  a 
perception  of  slower  egospeed.  A  second  possibility  is  that 
observers  simply  could  not  maintain  a  precise  memory 
representation  of  their  initial  egospeed,  and  over  time  their 
egospeed  drifted  upward,  perhaps  to  alleviate  boredom  with 
the  experimental  task.  Either  way,  it  is  clear  that  observers 
were  unable  to  maintain  constant  egospeed  based  on  optical 
flow  alone  and  further,  were  unaware  of  significant  increases 
in  egospeed  over  the  course  of  a  trial — sometimes  greater  than 
40%.  This  gross  insensitivity  to  visually  accelerating  self- 
motion  is  consistent  with  previous  psychophysical  research 
that  found  observers  to  be  relatively  insensitive  in 
discriminating  constant  velocity  from  accelerating  self-motion 
based  solely  on  visual  stimulation  (Dyre,  1997)  and  is  an 
important  design  consideration  for  virtual  displays  of 
egospeed.  Clearly,  designers  cannot  expect  pilots  to 
accurately  control  egospeed  by  maintaining  the  absolute  speed 
perceived  from  an  optical  flow  display.  A  better  solution 
might  be  to  use  virtual  displays  of  optical  flow  to  convey 
deviations  in  egospeed  from  a  target  speed  (egospeed-error). 
Because  displays  like  this  would  have  zero  movement  when 
observers  are  moving  at  the  target  speed,  motion  adaptation 
effects  would  be  eliminated.  The  use  of  optical  flow  for 
conveying  errors  in  speed  will  be  discussed  in  more  detail  in 
the  section  titled,  “Research  on  virtual  HUDs  for  speed 
control.” 

Motion  Parallax  and  Biases  in  Speed  Perception.  Factors 
other  than  variations  in  display  density  can  also  lead  to 
misperceptions  or  illusions  of  speed  and  egospeed.  For 
example,  observers  have  been  shown  to  mistake  changes  in 
the  variability  of  image  speed  as  changes  in  mean  speed 
(Atchley  &  Andersen,  1995).  Variability  in  image  speed 
occurs  whenever  an  observer  moves  through  an  environment 
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Figure  17.  Pictorial  representation  of  the  stimuli  used  to  examine 
the  relationship  between  perceived  egospeed  and  degree 
of  motion  parallax.  Left  Panels:  The  optical  flow  fields 
presented.  Each  arrow  represents  the  velocity  of  a 
moving  dot.  Right  Panels:  The  interplane  distances 
used  to  manipulate  the  degree  of  motion  parallax 
corresponding  to  the  optical  flows  shown  in  the  left 
panels. 

with  objects  located  at  different  distances.  Discounting  the 
effects  of  eye  movements,  objects  nearest  the  observer  project 
the  highest  image  velocities  and  objects  located  increasingly 
further  from  the  observer  project  increasingly  slower  image 
velocities.  This  variability  in  image  velocity  is  commonly 
known  as  motion  parallax  and  provides  information  about  the 
distance  to  objects  in  the  environment.  Motion  parallax  can 
also  serve  as  a  source  of  information  regarding  the  direction  of 
heading  (Cutting,  Springer,  Braren,  &  Johnson,  1992). 

Changes  in  the  magnitude  of  motion  parallax  or  image 
speed  variability  may  also  serve  as  an  important  cue  for 
estimating  egospeed.  For  the  sort  of  everyday  environments 
our  visual  system  has  evolved  to  perceive,  changes  in 
egospeed  result  in  changes  in  both  the  mean  and  variability  of 
image  speed.  As  a  result,  egospeed  could,  in  principle,  be 
estimated  from  either  of  these  statistical  moments  of  image 
speed.  Indeed,  as  shown  by  the  experiments  discussed 
previously,  mean  image  speed  is  an  important  factor 
underlying  perception  of  egospeed.  However,  the  influence  of 
image  speed  variability  has  not  been  systematically  assessed. 
Why  might  image  speed  variability  be  important?  One  reason 
is  that,  unlike  mean  image  speed,  variability  in  image  speed  is 
unaffected  by  eye  movements  and  thus  could  be  used  to 
perceive  egospeed  even  when  observers  are  moving  their  eyes 
to  track  objects  during  observer  movement  (McDevitt, 
Eggleston,  &  Dyre,  1999). 

A  series  of  experiments  was  conducted  to  more  fully 
understand  the  influence  of  motion  parallax  on  perceived 
egospeed.  These  experiments  measured  judgments  of 
egospeed  based  on  displays  in  which  the  variability  in  image 
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Figure  18.  Mean  perceived  egospeed  as  a  fiinction  of  interplane 
distance  (degree  of  motion  parallax).  Note:  the  mean 
image  flow  rate  of  the  displays  was  constant  across  all 
interplane  distances. 


speed  due  to  motion  parallax  was  manipulated  independently 
of  the  mean  image  speed  of  the  display.  This  dissociation 
between  mean  image  speed  and  variability  of  image  speed  was 
achieved  by  simulating  observer  movement  parallel  to  a  set  of 
three  parallel  planes  of  dots.  Constant  mean  image  speed  was 
obtained  by  maintaining  a  constant  distance  between  the  eye 
point  and  the  center  plane  of  the  three  planes  and  keeping  the 
distances  between  the  center  plane  and  each  of  the  two  outer 
planes  equal  (see  Figure  17).  Variability  in  image  speed  was 
manipulated  by  varying  the  distance  between  the  two  outer 
planes  and  the  center  plane,  what  will  subsequently  be  referred 
to  as  the  interplane  distance.  Increasing  interplane  distance 
increases  the  variability  in  image  speed  without  affecting 
mean  image  speed.  The  series  of  experiments  used  one  of  two 
methods  to  assess  the  influence  of  interplane  distance  on 
egospeed  estimates.  Some  experiments  required  observers  to 
make  magnitude  estimates  of  egospeed  in  response  to  the 
displays.  Other  experiments  serially  presented  a  pair  of  two 
displays:  a  standard  display  with  a  constant  speed  and 
interplane  distance,  and  a  comparison  display  where  speed  and 
interplane  distance  varied.  Observers  then  indicated  which 
display  produced  a  perception  of  faster  egospeed. 

The  general  result  found  with  both  methods  was  that 
increases  in  interplane  distance,  and  hence  increases  in 
variability  in  image  speed,  increased  judgments  of  perceived 
egospeed,  even  when  the  mean  image  speed  of  the  displays 
remained  constant  (see  Figure  18),  Similar  to  the  effects  of 
texture  density  on  perceived  speed,  the  increase  in  perceived 
speed  due  to  increased  variability  was  modest  compared  to  the 
effect  of  increasing  mean  image  speed.  However,  perceived 
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speed  was  found  to  be  affected  by  changes  in  image  speed 
variability  regardless  of  whether  observers  freely  scanned  the 
displays  or  fixated  a  stationary  cross-hair  located  in  the  center 
of  the  displays.  The  effect  of  image  speed  variability  was  also 
found  with  non-planar  environments  where  objects  were 
continuously  distributed  between  the  upper  and  lower  planes 
(a  volume  of  dots);  hence  it  appears  that  these  effects  cannot 
be  explained  by  observers  tracking  only  the  closest  plane, 
either  with  the  eye  or  an  attentional  focus.  The  results  taken 
as  a  whole  suggest  that  variability  in  image  velocity  affects  the 
integration  of  a  speed  signal,  perhaps  in  a  manner  related  to 
the  decreased  thresholds  for  relative  as  compared  to  absolute 
motion  (Gogel  &  McNulty,  1983). 

This  phenomenon  has  important  consequences  for  the 
design  of  virtual  displays  that  use  optical  flow  to  convey 
egospeed  information.  Designers  could,  for  example,  use 
motion  parallax  to  increase  the  perceived  speed  of  a  display, 
or  to  provide  speed  information  that  is  invariant  during  eye 
movements.  In  the  following  section  research  on  a  virtual 
display  for  egospeed  based  on  these  principles  is  described. 

Summary  and  Conclusions  of  Research  on  Egospeed 
Perception  and  Control  What  are  the  implications  of  the 
research  on  egospeed  perception  described  above?  First,  this 
research  suggests  that  perception  of  egospeed  is  based 
primarily  on  the  mean  image  speed  presented  in  a  display, 
although  it  may  also  be  influenced  by  other  factors  such  as 
texture  density  and  the  magnitude  of  motion  parallax.  These 
factors  would  need  to  be  carefully  considered  when  designing 
virtual  displays  that  use  optical  flow  to  convey  speed 
information. 

Second,  this  research  suggests  that  maintaining  a  constant 
egospeed  based  on  the  speed  of  optical  flow  is,  at  best 
difficult,  at  worst  impossible.  Observers  tasked  with 
maintaining  constant  speed  in  response  to  a  simulated 
headwind  disturbance  performed  poorly,  and  increased  their 
egospeed  over  time,  sometimes  by  as  much  as  80%  over  a 
one-minute  trial.  This  difficulty  in  maintaining  constant  speed 
could  result  from  a  combination  of  perceptual  adaptation, 
memory  drift,  and  the  general  human  insensitivity  to  visual 
accelerations.  Regardless  of  the  cause,  this  result  severely 
limits  the  utility  of  virtual  displays  that  code  absolute 
egospeed  by  the  absolute  speed  of  optical  flow,  because  over 
time  egospeed  will  increasingly  be  misperceived.  This  does 
not  mean,  however,  that  virtual  displays  using  optical  flow 
cannot  benefit  speed  control.  Virtual  displays  that  code  the 
deviation  of  egospeed  from  a  target  speed  could  be  potentially 
useful,  because  they  would  limit  perceptual  adaptation  (no 
movement  would  be  visible  when  the  pilot  is  at  the  target 
speed),  and  provide  a  memorable  target  speed,  zero  flow.  The 
following  section  describes  research  that  compared  such  a 
display  to  the  speed  indicator  specified  by  the  military 
standard  MIL-STD-1787B. 

Research  on  Virtual  HUDs  for  speed  control 

To  more  directly  test  the  potential  of  peripherally-located 
virtual  displays,  two  experiments  were  conducted  that 
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Figure  19.  Military  standard  HUD  based  on  MIL-STD-1787B  used 
as  a  baseline  of  comparison  for  the  virtual  HUD  (see 
Figure  20).  The  dial  on  the  left  indicates  airspeed.  The 
dial  on  the  right  indicates  altitude.  A  heading  indicator, 
pitch  ladder,  bank  indicator,  flight  path  indicator  and 
artificial  horizon  are  also  included. 

compared  a  peripherally-located  virtual  display  that  codes 
speed  error  using  optical  flow  to  the  speed  indicator  specified 
by  the  HUD  MIL-STD-1787B  (see  Fi^re  19).  The  design  of 
the  virtual  speed  indicator  (shown  in  Figure  20)  was  based  on 
the  general  principles  of  egospeed  perception  discovered  in 
the  course  of  the  basic  research  described  above.  Unlike  the 
military  standard  HUD  speed  indicator,  which  requires  direct 
fixation  of  gaze,  focal  attention,  and  higher  cognitive 
resources  to  read  and  interpret,  the  virtual  speed  error 
indicator  was  designed  to  be  processed  by  peripheral  vision, 
with  little  or  no  attentional  allocation  or  interpretation 
required.  In  theory,  this  type  of  display  should  provide  better 
speed  control  while  the  pilot  is  under  high  mental  workload, 
because  it  provides  pilots  with  speed  information  in  a 
ecological  format  compatible  with  naturally-evolved  orienting 
and  motion  coding  processes  that  do  not  demand  large  pools 
of  attentional  resources. 

How  should  such  a  display  be  designed  to  best  take 
advantage  of  human  visual  processing?  The  virtual  speed 
indicator  tested  here  consisted  of  optical  flow  fields  defined  by 
moving  arrows  presented  to  large  areas  (25  x  34  degrees,  H  x 
V)  of  the  left  and  right  peripheral  visual  fields  (see  Figure  20). 
The  peripheral  visual  system  is  particularly  well-suited  for 
sensing  motion  distributed  over  a  wide  area  (Tanaka,  1998) 
and  can  dominate  processes  of  spatial  orientation  (Brandt  et 
al,  1973).  Further,  peripheral  vision  is  relatively  underused 
by  current  display  systems  that  are  typically  confined  to  the 
central  20  degrees  of  the  visual  field.  Because  speed  control  is 
generally  independent  of  flight-path  control  (Haskell  & 
Wickens,  1993),  locating  speed  information  in  a  peripheral 
display  that  is  spatially  removed  from  displays  supporting 
flight-path  control  (such  as  the  artificial  horizon  and  heading 
display)  is  compatible  with  the  proximity-compatibility 
principle  (Carswell  &  Wickens,  1987),  which  states  that 
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Figure  20.  Virtual  HUD  based  on  MIL-STD-1787B  in  which  the 
airspeed  indicator  dial  has  been  replaced  by  the  virtual 
speed-error  display  represented  by  the  peripherally- 
located  moving  arrows.  All  other  display  elements  were 
identical  to  the  HUD  shown  in  Figure  19. 

displays  supporting  independent  tasks  should  be  separated 
while  those  supporting  interdependent  tasks  should  be 
integrated. 

The  size,  speed,  orientation,  and  direction  of  the  moving 
arrows  represented  the  magnitude  and  direction  of  egospeed 
error  in  the  virtual  HUD.  The  direction  of  error  (above  or 
below  target  speed)  was  coded  in  the  orientation  and 
movement  direction  of  the  arrows.  Positive  egospeed  errors 
corresponding  to  pilots  moving  faster  than  their  target  speeds 
were  represented  by  moving  the  arrows  in  depth  along 
trajectories  parallel  to  the  line  of  sight,  with  the  orientation  of 
the  arrow  aligned  with  the  direction  of  movement,  i.e.,  aimed 
toward  the  observer.  The  arrows  thus  created  an  expanding 
pattern  of  optical  flow  radiating  outward  from  a  direction 
straight  ahead.  For  negative  speed  errors  (egospeed  below  the 
target  speed)  these  directions  were  reversed:  the  motion  and 
orientation  of  the  arrows  were  directed  away  from  the 
observer,  which  created  a  contracting  pattern  of  optical  flow 
converging  on  the  straight-ahead  direction.  For  both  positive 
and  negative  speed  errors,  the  direction  indicated  by  the 
orientation  and  movement  of  the  arrows  was  consistent  with 
the  direction  of  throttle  movement  necessary  to  correct  the 
error.  This  stimulus-response  compatibility  takes  advantage 
of  natural  perceptual-motor  mappings  (Wickens  &  Hollands, 
1999),  and  should  minimize  the  mental  workload  associated 
with  mapping  the  appropriate  control  movement  to  the 
information  presented  in  the  display. 

The  magnitude  of  speed  error  was  coded  by  arrow  size 
and  speed  of  movement.  The  speed  of  the  moving  arrows 
increased  linearly  with  the  magnitude  of  speed  error.  The  size 
of  the  moving  arrows  increased  in  discrete  steps  as  the 
magnitude  of  speed  error  increased.  When  egospeed  equaled 
the  target  speed,  the  arrows  had  zero  size  and  moved  at  zero 


velocity;  in  effect,  the  arrows  disappeared  from  view.  As 
speed  error  increased  from  zero  beyond  a  threshold  value,  a 
field  of  slowly  moving  arrows  would  suddenly  appear.  This 
sudden  onset  briefly  captures  attention  and  informs  the  pilot 
that  the  speed  error  is  too  large.  If  the  pilot  reacts  swiftly  to 
the  motion  and  orientation  of  the  arrows  and  reduces  the  speed 
error  below  the  threshold  value,  the  arrows  would  once  again 
disappear.  If,  on  the  other  hand,  the  pilot  does  not 
immediately  respond  and  speed  errors  continue  to  increase  (as 
might  happen  during  maneuvers  in  which  attentional  load  is 
high),  the  arrows  move  progressively  faster  and  step  upward 
in  size.  The  progressively  faster  arrow  movement  and 
additional  sudden  onsets  associated  with  each  step  increase  in 
size  increase  the  probability  that  attention  will  be  briefly 
captured  by  the  display  and  appropriate  control  responses 
made.  Hence,  the  saliency  of  the  display  adapts  to  the  severity 
of  the  error  in  egospeed  control.  Large  errors  produce  an 
extremely  salient  display  that  is  difficult  to  ignore,  while  small 
errors  result  in  the  display  disappearing  entirely. 

To  evaluate  this  display,  two  experiments  were  conducted 
that  measured  the  precision  of  egospeed  and  flight-path 
control  during  simulated  flight.  Two  types  of  speed  indicators 
were  examined:  the  Mil-Standard  (MIL-STD-1787B)  airspeed 
indicator  and  the  virtual  speed-error  indicator  described  above 
(see  Figures  19  and  20,  respectively).  Participants  with  no 
flight  experience  other  than  video  games  flew  a  simulated 
aircraft  with  simplified  control  dynamics  through  a  simulated 
environment.  The  simulated  environment  consisted  of  two  flat 
planes  consisting  of  green,  yellow,  and  brown  squares, 
separated  in  elevation  by  a  cliff  composed  of  grey  squares  of 
varying  shades.  To  provide  additional  texture  in  the 
environment,  each  plane  was  covered  with  simulated  trees, 
scattered  randomly  across  the  planes.  Fog  effects  were  used  to 
enrich  perceived  depth  in  the  displays  through  an  aerial 
perspective  cue.  A  flight-path  was  defined  by  waypoints 
consisting  of  wireframe  squares  floating  above  the  planes.  A 
target  speed  and  altitude  for  each  leg  of  the  flight  path  was 
defined  for  each  waypoint  and  displayed  in  the  HUDs.  The 
location  of  the  waypoints,  as  well  as  the  speed  and  altitude 
targets  they  defined,  varied  randomly  from  trial  to  trial  around 
two  general  flight  paths.  Participants  were  instructed  to  attain 
the  target  speeds  and  altitudes  defined  by  each  waypoint  as 
soon  as  possible,  and  maintain  them  until  they  had  passed 
through  the  next  waypoint.  To  induce  participants  to  make 
continuous  corrective  control  input,  simulated  wind 
disturbances  defined  by  sum-of-sine  functions  continuously 
changed  forward  speed,  altitude,  pitch,  yaw,  and  roll. 

Participants  controlled  their  movement  through  the 
environment  by  moving  a  joystick  with  their  right  hand,  a 
throttle  with  their  left  hand,  and  rudder  pedals  with  their  feet. 
Forward-backward  movements  of  the  joystick  controlled  the 
angular  velocity  of  changes  in  pitch,  while  side-to-side 
movements  controlled  the  angular  velocity  of  changes  in  roll. 
Movement  of  the  rudder  pedals  controlled  the  velocity  of  yaw 
rotations,  and  movement  of  the  throttle  controlled  the 
acceleration  of  forward  movement.  These  control  movements 
were  recorded  and  analyzed,  together  with  the  simulated 
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Figure  21.  RMS  speed  error  (top)  and  mean  altitude  error  (bottom) 
for  single  and  dual  task  control  with  HUDs  containing 
either  the  MIL-STD-1787B  dial  airspeed  indicator  or  the 
virtual  speed-error  indicator.  Error  bars  represent 
standard  errors  of  the  mean. 

aircraft’s  location,  altitude,  and  speed  as  well  as  the  target 
speeds  and  altitudes  set  by  the  waypoints.  In  addition  to  the 
performance  measures,  these  experiments  also  measured 
subjective  mental  workload  using  the  NASA-TLX  and  gaze 
direction  using  an  eye-head  tracking  system. 

To  compare  the  relative  difficulty  and  mental  workload  of 
controlling  speed  and  altitude  with  the  Mil-standard  and 
virtual  HUDs,  performance,  workload,  and  gaze  direction 
were  measured  for  both  single-  and  dual -task  conditions.  For 
single-task  egospeed-control  trials,  participants  needed  only  to 
control  egospeed  with  the  throttle  while  an  autopilot 
controlled  pitch,  yaw,  and  roll  to  maintain  the  target  flight 
path  and  altitude.  For  single-task  flight-path  control  trials, 
participants  controlled  pitch,  yaw,  and  roll  with  the  joystick 
and  rudder  pedals  to  guide  themselves  along  the  proper  flight 
path,  but  the  autopilot  controlled  the  throttle  governing 
egospeed.  For  dual-task  trials,  participants  used  all  the  input 
devices  to  control  both  flight  path  and  egospeed.  The  virtual 
speed-error  display  was  expected  to  provide  the  largest 
performance  benefit  for  the  dual-task  conditions,  because 
successful  performance  of  the  dual-task  requires  participants 
to  divide  their  attention  between  egospeed  control  and  flight- 
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Figure  22.  Gaze  distributions  (fixation  dwell  times)  averaged  across 
all  subjects  and  trials  for  the  dual  control  task  with  the 
MIL-STD-1787B  airspeed  dial  (top)  and  the  virtual 
speed-error  indicator  bottom).  The  shading  indicates 
the  percentage  of  time  gaze  dwelled  within  a  2  degree 
radius  of  a  particular  display  coordinate. 


path  control.  In  contrast,  for  the  single-task  conditions 
participants  could  devote  all  their  mental  resources  to 
controlling  speed  or  flight-path,  providing  an  estimate  of  the 
optimal  performance  attainable  with  a  given  display. 

The  results  of  this  study  show  that  the  virtual  speed-error 
HUD  did  indeed  reduce  mental  workload  and  increase  the 
precision  of  egospeed  and  altitude  control  relative  to  the  mil- 
standard  airspeed  indicator.  The  speed  and  altitude  control 
results  are  presented  in  the  top  and  bottom  panels  of  Figure 
21,  respectively.  Note  that  for  single-task  conditions,  altitude 
and  speed  control  were  equivalent  for  the  two  displays,  which 
suggests  that  both  types  of  HUD  can  provide  adequate  speed 
information  when  attention  is  fully  devoted  to  processing  the 
display.  However,  for  dual-task  conditions,  altitude  and  speed 
control  were  more  accurate  with  the  virtual  speed-error  HUD. 
Estimates  of  subjective  mental  workload  were  consistent  with 
the  performance  data,  participants  rated  the  virtual  speed  error 
indicator  as  inducing  less  mental  workload  than  the  military 
standard  speed  indicator. 

The  gaze  patterns  associated  with  each  display  confirm 
the  difference  in  visual  resources  demanded  by  each  display 
(see  Figure  22).  Participants  did  indeed  process  the  virtual 
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HUD  with  peripheral  vision,  which  enabled  them  to  direct  a 
greater  proportion  of  their  gaze  time  to  information  important 
for  controlling  flight-path  as  compared  to  the  mil-standard 
HUD,  which  required  direct  fixation  of  gaze  to  read. 

Taken  together,  the  performance  and  gaze  distribution 
results  show  that  for  untrained  observers  flying  a  simplified 
flight  simulation,  speed  and  flight-path  control  is  better  served 
by  peripherally  located  virtual  displays  that  take  advantage  of 
the  natural  orienting  processes  in  which  peripheral  vision 
excels. 

These  results  have  important  implications  for  the  design 
of  future  HUDs.  Clearly,  the  peripheral  visual  fields  are  an 
important  visual  resource  that  is  underused  with  current  HUDs 
and  may  be  a  particularly  valuable  resource  for  processing 
flight  parameters  related  to  spatial  orientation,  like  control  of 
airspeed.  Virtual  displays  of  these  parameters  may  provide 
pilots  with  information  in  a  natural  manner  that  minimizes  the 
impact  on  pilot  workload  and  central  visual  field  processing, 
thereby  freeing  resources  for  processing  other  flight 
information  and  increasing  flight  safety.  Whether  these 
advantages  hold  for  experienced  pilots  in  real  flight  situations 
needs  to  be  evaluated  with  further  research. 
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